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Preface 



The Department of Homeland Security (DHS) Office of Inspector General (OIG) was 
established by the Homeland Security Act of 2002 (Public Law 107-296) by amendment 
to the Inspector General Act of 1978. This is one of a series of audit, inspection, and 
special reports prepared as part of our oversight responsibilities to promote economy, 
efficiency, and effectiveness within the department. 

This report addresses the strengths and weaknesses of the Transportation Security 
Administration's procedures, equipment, and supervision to ensure that Transportation 
Security Officers are able to prevent threat items from being introduced into the checked 
baggage systems of the nation's airports. It is based on interviews with Transportation 
Security Officers and Transportation Security Administration officials, direct 
observations, field testing, and a review of applicable documents. 

The recommendations herein have been developed to the best knowledge available to our 
office, and have been discussed in draft with those responsible for implementation. We 
trust this report will result in more effective, efficient, and economical operations. We 
express our appreciation to all of those who contributed to the preparation of this report. 




Richard L. Skinner 
Inspector General 



OIG 

Department of Homeland Security 
Office of Inspector General 

Background 

The Transportation Security Administration (TSA) is responsible 
for overseeing aviation security and ensuring the safety of the air 
traveling public. This includes the screening of checked baggage 
and safeguarding it from prohibited items. We conducted 
unannounced, covert testing to evaluate how effective the On- 
Screen Alarm Resolution Protocol (OSARP) certified operators are 
at determining whether an object causing an explosives detection 
system to alarm poses a serious threat and how those threats are 
being resolved. 

We conducted unannounced, covert testing at eight domestic 
airports, from April 2008 through June 2008. Because TSA has 
deployed explosives trace detection machines at most of the major 
domestic airports for operational use, we included locations that 
were using these technologies in conjunction with trained 
Transportation Security Officers (TSOs) utilizing the OSARP 
when we made our airport selections. 

Our purpose in testing was to determine whether: (1) TSA's 
implementation of OSARP is effective; (2) TSA's checked 
baggage screening procedures and standards are adequate; 
(3) TSOs are following those screening procedures and standards; 
and (4) aviation security screening equipment and technologies are 
functioning properly and as intended to prevent threat items from 
entering the checked baggage system. 
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Results of Audit 



The number of tests conducted, the names of the airports tested, 
and the quantitative and qualitative results of our testing are 
classified. We have shared that information with the department, 
the Transportation Security Administration, and appropriate 
congressional committees. 

We identified vulnerabilities in the security of checked baggage at 
the eight domestic airports where we conducted testing. As a 
result of our testing, we made four recommendations. TSA 
concurred with three of the four recommendations and concurred 
in part with the other recommendation. When fully implemented, 
these recommendations should strengthen the checked baggage 
screening process. 

We gave a copy of our draft report to TSA for review and 
comment. The unclassified portion of TSA's response to our draft 
report is summarized below. 

TSA's Response 

TSA appreciates the work done by OIG on this engagement, and 
can generally validate OIG's data against TSA's own covert 
testing programs when TSA analysis rules are applied. While we 
disagree over OIG's characterizations of Explosives Detection 
System (EDS) performance and have excluded them from our 
analysis, the remainder of OIG's data is in line with TSA test 
results in both agency-run covert testing programs. 

OIG's data continues to demonstrate the effectiveness of TSA's 
checked baggage security system in detecting and removing threat 
objects. 

TSA Covert Assessment Programs Have Similar Results 

Aviation Screening Assessment Program 

TSA's Aviation Screening Assessment Program (ASAP) is 
designed to measure screening performance using a realistic 
standardized test scenario framework. Through ASAP, TSA 
conducts covert testing of the TSA screening process. The primary 
purpose is to provide statistical data to identify points of and 
reasons for failures within the checkpoint and checked baggage 
screening process. The data generated from ASAP testing is used 
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to make national recommendations to improve screening 
performance. 

TSA Office of Inspection Testing 

TSA's Office of Inspection has a very robust testing program 
designed to identify systemic vulnerabilities in transportation 
security systems. Through the Office of Inspection testing 
program, subject matter experts develop and test specific 
hypotheses regarding potential system vulnerabilities. These tests 
are not designed to be performance measures. Rather, they are 
evaluations of system vulnerabilities that can be used to design 
countermeasures. When viewed in this light, the qualitative results 
from these experiments are highly valuable in analyzing 
vulnerabilities, with conclusions from these experiments informing 
decisions at the strategic level. 

TSA appreciates the work done by OIG on this engagement, which 
further validates TSA's own testing of the checked baggage 
screening system effectiveness. 
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Appendix A 

Major Contributors to this Report 

Patrick O'Malley, Director, Transportation Security Division 

James Yeager, Program Manager 

Cecilia Barela, Analyst-in-Charge 

Sharon Trodden, Senior Auditor 

Danielle de Sanctis, Program Analyst 

Barry Cleveland, Program Analyst 

Vanessa Santos, Independent Referencer 
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Appendix B 
Report Distribution 



Department of Homeland Security 

Secretary 

Acting Deputy Secretary 

Chief of Staff for Operations 

Chief of Staff for Policy 

Acting General Counsel 

Executive Secretary 

Director, GAO/OIG Liaison Office 

Assistant Secretary for Office of Policy 

Assistant Secretary for Office of Public Affairs 

Assistant Secretary for Office of Legislative Affairs 

Acting Assistant Secretary, Transportation Security Administration 

DHS OIG Liaison, TSA 

Office of Management and Budget 

Chief, Homeland Security Branch 
DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees, as appropriate 
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ADDITIONAL INFORMATION AND COPIES 

To obtain additional copies of this report, please call the Office of Inspector General (OIG) at (202) 254-4199, 
fax your request to (202) 254-4305, or visit the OIG web site at www.dhs.gov/oig. 



OIG HOTLINE 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of criminal or noncriminal 
misconduct relative to department programs or operations: 

• Call our Hotline at 1-800-323-8603; 

• Fax the complaint directly to us at (202) 254-4292; 

• Email us at DHSOIGHOTLINE@dhs.gov; or 

• Write to us at: 

DHS Office of Inspector General/MAIL STOP 2600, 
Attention: Office of Investigations - Hotline, 
245 Murray Drive, SW, Building 410, 
Washington, DC 20528. 



The OIG seeks to protect the identity of each writer and caller. 



